A common research interest in ubiquitous computing has been the development of inexpensive and easy-to-deploy sensing systems that support activity detection and context-aware applications in the home. For example, several researchers have explored using arrays of low-cost sensors, such as motion detectors or simple contact switches. Such sensors are discussed by Tapia, E. M., et al. “Activity recognition in the home setting using simple and ubiquitous sensors,” Proc of PERVASIVE 2004, pp. 158-175, Tapia, E. M., et al. “The design of a portable kit of wireless sensors for naturalistic data collection,” Proc of Pervasive 2006, pp. 117-134, and Wilson, D. H., et al. “Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors,” Proc of Pervasive 2005, pp. 62-79, 2005, for example.
Although these solutions are cost-effective on an individual sensor basis, they are not without some drawbacks. For example, having to install and maintain many sensors may be a time-consuming task, and the appearance of many sensors may detract from the aesthetics of the home. This is discussed by Beckmann, C., et al. “Some Assembly Required: Supporting End-User Sensor Installation in Domestic Ubiquitous Computing Environments,” Proc of Ubicomp 2004. pp. 107-124. 2004, and Hirsch, T., et al. “The ELDer Project: Social, Emotional, and Environmental Factors in the Design of Eldercare Technologies,” Proc of the ACM Conference on Universal Usability, pp. 72-79, 2000, for example.
In addition, the large number of sensors required for coverage of an entire home may increase the number of potential failure points. To address these concerns, recent work has focused on sensing through existing infrastructure in a home. For example, researchers have looked at monitoring the plumbing infrastructure in the home to infer basic activities or using the residential power line to provide indoor localization. See, for example, Fogarty, J., et al. “Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition,” Proc of ACM Symposium on User Interface Software and Technology (UIST 2006) 2006, and Patel, S. N., et al. “PowerLine Positioning: A Practical Sub-Room-Level Indoor Location System for Domestic Use,” Proceedings of Ubicomp 2006.
Research relating to activity and behavior recognition in a home setting may be classified by examining the origin of the sensing infrastructure disclosed herein. The first area of classification includes approaches that introduce new, independent sensors into the home that directly sense various activities of its residents. This classification includes infrastructures where a new set of sensors and an associated sensor network (wired or wireless) is deployed. A second area encompasses those approaches that take advantage of existing home infrastructure, such as the plumbing or electrical busses in a home, to sense various activities of residents. The goal of the second approach is to lower the adoption barrier by reducing the cost and/or complexity of deploying or maintaining the sensing infrastructure.
Some research approaches use high-fidelity sensing to determine activity, such as vision or audio systems that capture movement of people in spaces. See, for example, Bian, X., et al. “Using Sound Source Localization in a Home Environment,” Proc of the International Conference on Pervasive Computing, 2005, and Koile, K., et al. “Activity Zones for Context-Aware Computing,” Proc of UbiComp 2003: Ubiquitous Computing, 2003, Seattle, Wash., USA.
Chen et al. in “Bathroom Activity Monitoring Based on Sound,” Proc of Pervasive 2005, pp. 47-61, 2005, installed microphones in a bathroom to sense activities such as showering, toileting, and hand washing. While these approaches may provide rich details about a wide variety of activities, they are often very arduous to install and maintain across an entire household.
Use of these high fidelity sensors in certain spaces raise concerns about the balance between value-added services and acceptable surveillance, particularly in home settings. This is discussed by Beckmann, C., et al. “Some Assembly Required: Supporting End-User Sensor Installation in Domestic Ubiquitous Computing Environments,” Proc of Ubicomp 2004, pp. 107-124. 2004, Hirsch, T., et al. “The ELDer Project: Social, Emotional, and Environmental Factors in the Design of Eldercare Technologies,” Proc of the ACM Conference on Universal Usability, pp. 72-79, 2000, and Iachello, G., et al. “Privacy and Proportionality: Adapting Legal Evaluation Techniques to Inform Design In Ubiquitous Computing,” Proc of CHI 2005, pp. 91-100, 2005.
Another class of approaches explores the use of a large collection of simple, low-cost sensors, such as motion detectors, pressure mats, break beam sensors, and contact switches, to determine activity and movement. See Tapia, E. M., et al. “Activity recognition in the home setting using simple and ubiquitous sensors,” Proc of PERVASIVE 2004, pp. 158-175, 2006, Tapia, E. M., et al. “The design of a portable kit of wireless sensors for naturalistic data collection,” Proc of Pervasive 2006, pp. 117-134, and Wilson, D. H., et al. “Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors,” Proc of Pervasive 2005, pp. 62-79, 2005. The Tapia et al. papers discuss home activity recognition using many state change sensors, which were primarily contact switches. These sensors were affixed to surfaces in the home and logged specific events for some period of time. The advantage of this approach is being able to sense physical activities in a large number of places without the privacy concerns often raised for high-fidelity sensing (e.g., bathroom activity). There are also some disadvantages to this add-on sensor approach, which include the requirements of powering the sensors, providing local storage of logged events on the sensor itself, or a wireless communication backbone for real-time applications. These requirements all complicate the design and maintenance of the sensors, and the effort to install many sensors and the potential impact on aesthetics in the living space may also negatively impact mass adoption of this solution.
As an example of the often difficult balance of the value of in home sensing and the complexity of the sensing infrastructure, the Digital Family Portrait is a peace of mind application for communicating well-being information from an elderly person's home to a remote caregiver. See, for example, Rowan, J. et al. “Digital Family Portrait Field Trial: Support for Aging in Place,” Proc of the ACM Conference on Human Factors in Computing Systems (CHI 2005), pp. 521-530, 2005. In this deployment study, movement data was gathered from a collection of strain sensors attached to the underside of the first floor of a home. Installation of these sensors was difficult, time-consuming, and required access under the floor. Although the value of the application was proven, complexity of the sensing limited the number of homes in which the system could be easily deployed.
Other approaches, which are similar to ours, are those that use existing home infrastructure to detect events. Fogarty et al. “Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition,” Proc of ACM Symposium on User Interface Software and Technology (UIST 2006), 2006, explored attaching simple microphones to a home's plumbing system, thereby leveraging an available home infrastructure. The appeal of this solution is that it is low-cost, consists of only a few sensors, and is sufficient for applications, such as the Digital Family Portrait, for which the monitoring of water usage is a good proxy for activity in the house. This approach requires relatively long timescales over which events must be detected, sometimes up to ten seconds. This longer time increases the likelihood of overlapping events, which are harder to distinguish.
In contrast, power line event detection operates over timescales of approximately half a second and thus overlapping is less likely. Some water heaters constantly pump hot water through the house, complicating the detection of some on-demand activities. Detecting noise on water pipes introduced by other household infrastructure requires careful placement of the microphone sensors. Some homes may not have plumbing infrastructure that is easily accessible, particularly those with a finished basement or no basement at all. Despite these limitations, this solution is very complementary to our approach, as some events revealed by water usage, such as turning on a faucet in a sink or flushing a toilet, do not have direct electrical events that could serve as predictive antecedents. The converse also holds, as a light being turned on often does not correlate with any water-based activity.
Another “piggybacking” approach is to reuse sensing infrastructure or devices in the home that may be present for other purposes. For example, ADT Security System's QuietCare offers a peace of mind service that gathers activity data from the security system's motion detectors. This is discussed on the ADT QuietCare website at http://www.adt.com/quietcare/.
There are other techniques that employ electrical power use to sense activity. For example, some researchers have monitored electrical current flow to infer the appliances or electrical equipment being used in the house as a proxy for detecting activity. See, for example, Paradiso, J. A. “Some Novel Applications for Wireless Inertial Sensors,” Proc of NSTI Nanotech 2006, Vol. 3, Boston, Mass., May 7-11, 2006, pp. 431-434, and Tapia, E. M., et al. “The design of a portable kit of wireless sensors for naturalistic data collection,” Proc of Pervasive 2006, pp. 117-134.
The Paradiso platform monitors current consumption of various appliances of interest. Changes in current flow indicate some change in state for the instrumented appliance, such as a change from on to off. This solution requires a current sensor to be installed inline with each appliance or around its power cord and thus only works well if it is sufficient to study the usage of a small subset of appliances and those appliances' power feeds are easy accessible. An extension to the Paradiso work is to install current sensors on major branch circuits of the power lines, but this may require professional installation to provide an acceptable level of safety. However, it would be desirable to detect a larger number of appliances with less instrumentation and with a much easier deployment phase.
There is therefore a need for apparatus and methods for detecting electrical device actuation using electrical noise over a power line.